program-constrained-language-model-sampling | program constraints on pre-trained language models

program-constrained-language-model-sampling is a Jupyter Notebook library. program-constrained-language-model-sampling has no bugs, it has no vulnerabilities, it has a Permissive License and it has low support. You can download it from GitHub.

The aim of this project is to investigate the use of program constraints on pre-trained language models to improve their ability to generate structured text as output. Language models can accurately predict the next token in a sequence. Unfortunately generated text is not necessarily structured in a way that can be utilised by other programs. For example, a language model that extracts json objects from unstructured text may fail to produce a valid json object by adding additional trailing commas or introducing comments. We propose a simple method to constrain the tokens that can be sampled by a language model by using an external program to decide which tokens in the vocabulary are valid at a given point in the sequence. We then force the language model to only sample over these valid tokens.